10-millisecond Computing

Despite computation becomes much complex on data with unprecedented large-scale, we argue computers or smart devices should and will consistently provide information and knowledge to human being in the order of a few tens milliseconds. We coin a new term 10-millisecond computing to call attention to this class of workloads. Public reports indicate that internet service users are sensitive to the service or job-level response time outliers, so we propose a very simple but powerful metric—outlier proportion to characterize the system behaviors. The outlier proportion is defined as follows: for N completed requests or jobs, if M jobs or requests’ latencies exceed the outlier limit t, e.g. 10 milliseconds, the outlier proportion is N . 10-millisecond computing raises many challenges for both software and hardware stacks. In this paper, as a case study we investigate the challenges raised for conventional operating systems. For typical latency-critical services running with Linux on a 40-core server—a main-stream server hardware system in near future, we found, when the outlier limit decreases, the outlier proportion of a single server will significantly deteriorate. Meanwhile, the outlier proportion is further amplified by the system scale, including the system core number. For a 1K-scale system, we surprisingly find that to reduce the service or job-level outlier proportion to 10%, running Linux (version 2.6.32) or LXC (version 0.7.5 ) or XEN (version 4.0.0), respectively, the outlier proportion of a single server needs to be reduced by 871X, 2372X, 2372X accordingly. We also conducted a list of experiments to reveal the current Linux systems still suffer from poor outlier performance, including Linux kernel version 3.17.4, Linux kernel version 2.6.35M, a modified version of 2.6.35 integrated with sloppy counters proposed by Boyd-Wickizer et al. and two representative real time schedulers.